Applying supervised learning on Scriabin

We can use this data on Scriabin’s music to try out some machine learning tasks. Since there is a general consensus on which period of Scriabin’s career each sonata belongs to, can this same consensus be extracted from just the data? This sounds like a classic supervised learning classification task, since we have labelled data which we want our model to predict.

We will try to predict the period of Scriabin’s career a track belongs to, based on the data Spotify’s API provides, by means of a random forest. To proceed however, we might need to take a couple factors into account first.

The 4 pianists’ recordings seemed to be sufficient for our analysis thus far, but when training these kinds of models, this small sample of recordings might not provide enough quality data. Our dataset is also quite unbalanced to begin with, containing very little middle period tracks compared to early ones. Not to mention how we also need to have extra data on hand that won’t be used during the training process in order to properly test our model. so for these reasons, it might not be a bad idea to extend our original corpus with more recordings. The Scriabin sonata recordings of the following pianists have been added:

This way, we have ample data to train our model with, while still being able to create a balanced test split. Otherwise the model could simply classify every recording to be an early work and still retain a relatively high accuracy.

Aside from that, we also need to decide which features the model will base it’s classifications on. Specifically for our corpus, the features Spotify provides often say way to much, such as duration, or say way to little, such as acousticness. This is simply due to the nature of our corpus and the nature of the features Spotify provides. I personally am most interested in seeing if the model is able to perform well using just the audio content of the tracks, and will therefore only provide the model with the pitch and timbre features.

Random forest results.

These are the results of fitting our model to the training data after fine tuning it’s hyper parameters.

Test results

Taking a closer look at the actual performance on the test set.

The style of Scriabin

The style of Scriabin

There are many composers who have made a name for themselves when it comes to their contribution to the piano repertoire. One such composer is Alexander Scriabin. However, what separates him from other composers of similar regard, is the way in which his musical language has developed throughout his life. The influence of Frederic Chopin and other romantic composers can clearly be discerned at the start of his career. In spite of that, his music would increasingly veer away from traditional harmony and tonality over the turn of the 20th century. Much like Arnold Schoenberg, he had developed a unique, modernist style of music that would influence many other 20th century composers.

One of my goals is to examine whether Scriabin’s changing style could be accurately represented. This development is perhaps best reflected in Scriabin’s 10 piano sonatas, which span the majority of his composing career. My corpus therefore consists of recordings of these sonatas by four pianists, namely Vladimir Ashkenazy, Maria Lettberg, Igor Zhukov and Dmitri Alexeev. These pieces, much like his other works, leave the performer with room for their own interpretation. So It would perhaps also be interesting to see in what capacity Spotify could measure the difference between performers.

When considering tracks of particular interest, the 4th and 5th piano sonatas come to mind. The former being one of his first pieces to display a concrete change in style, and the later being one of the first steps into truly new territory. Yet despite their differences they do both have a distinct touch of lightness that is in my opinion more difficult to find in his other sonatas. A further look into these pieces may yield intriguing results.

Comparing the energy and valence for all sonatas

This plot displays the valence and energy levels of the sonatas performed by Igor Zhukov, just as an example. If you focus on the valence feature, valence being a measure of positiveness, you might notice that while the differences across sonatas are small, all the late sonatas seem to lay at a distinct valence level compared to the rest.

Those familiar with the music in question might not be surprised by this. These pieces are very representative of Scriabin’s changing musical style, despite the fact that they are individually still unique. The valence feature is Spotify’s way of interpreting a piece’s ‘happiness’. Whether the late sonatas all contain the same degree of happiness, is debatable. It is nonetheless interesting how Spotify is confidentially able to differentiate between the late sonatas and the rest.

The late sonatas contain similar valence levels accros all performers

here you can see the previous results extend very well to other pianists and their recordings. While the energy and valence levels of the early and middle period of Scriabin’s works vary quite a bit, the late sonatas all fall on the same range of valence, even across different performers.

Whether these pieces are all equally unhappy compared to the rest of the sonatas, is debatable as I’ve mentioned before. I personally believe that some late sonatas are noticeably more restless in general than others and perhaps deserve a slightly difference valence score than the others. At the end of the day however, we simply don’t know the exact way Spotify assigns these scores, not to mention how it likely is not very well tuned to tonally ambiguous modernist piano pieces. Other than that, I think it’s save to say that Spotify does seem to do a good job at picking up the fact that the late sonatas sound fundamentally different. It even seems to place the 5th sonata quite often very near to the later ones, which definitely is more similar to those pieces than to the earlier sonatas.

At the same time, the pieces show somewhat of a variance in energy levels across performers. This could perhaps reflect the natural room for interpretation of Scriabin’s works, albeit this is hard to say from just this data.

A piece’s tempo can vary quite a bit accross pianists

When comparing different performances of the same pieces, many differences are bound to pop up. Such is the nature of interpretation, you could say. Two variables that are relatively easy to compare and measure are loudness and tempo. How accurate Spotify’s analysis actually is, remains difficult to determine, but it couldn’t hurt to take a look, right?

As expected, different pianist show quite a different spread when it comes to their tempo and tempo variance throughout their recordings. Maria Lettberg’s recordings seem to all be quite consistent, while Vladimir Ashkenazy takes a lot more liberty when it comes to tempo. Moreover, taking a look at the 7th sonata specifically for instance shows how Lettberg’s variance in tempo is exceptionally low compared to the other pianists. Besides that, Igor Zhukov’s recordings all seem to be consistently louder. This might also just be a result of the different recording processes.

As mentioned however, we should take these values with a grain of salt. When looking at the 7th sonata again, Ashkenazy’s mean bpm is around 70, while Alexeev’s is around 170. You can not take away that Alexeev is playing this piece twice as fast from these numbers however, since both recordings are around 12 minutes long.

How clearly does a chromagram show a sonata’s key?

The Spotify API allows us to retrieve a track’s pitch content. That data can be used to create a chromagram, which displays the distribution of energy in each pitch class over the course of the recording. If a piece were to clearly be in the key of A for instance, a chromagram of its recording is likely going to reflect this for instance.

Scriabin’s early sonatas are somewhat traditional when it comes to harmony. His pieces at this point of his career usually have an unambiguous tonal center. Take his 3rd and 4th sonatas for instance. These pieces are respectively in F#-minor and F#-major, and as expected, their chromagrams show how those pitches are fairly prevalent in both pieces. Especially at the end of each movement.

Tonality is identifyable in early works…

We can use a recording’s pitch content to try and estimate its key and which chords appear in it. An example of this can be seen here for the first movement of the second sonata in G# minor. This piece can quite easily be described in sonata form, with the exposition starting in G# minor as expected, briefly going to D# minor before largely remaining in B major.

The development section continues in B major, before quickly and briefly shifting to D major, G minor, C minor, F minor and eventually leads back to G# minor. The recapitulation continues on in G# minor, before quickly modulating to the relative E major and remaining there until the end of the movement.

These key changes are reflected quite well in the keygram, although it certainly helps to know them beforehand.

What about the later sonatas?

There’s a lot that can be said about Scriabin’s harmonic language. One important aspect of his later works is how, to a fair extent, it’s based on unique scales and chords such as the octatonic scale or the ‘mystic’ chord. I’ve tried to create a mask of one of the organizations of the octatonic scale for each pitch class to see if it may yield interesting results for the 9th sonata.

It is admittedly quite hard to say whether these results hold any weight. The plot does seem to consistently mark certain sections to use a certain octatonic scale. I’ve looked through the score and confirmed that some of these sections do indeed use the pitches of these octatonic scales. However, the same cannot be said for the other sonatas, some of which also do make use of this scale. There’s also the fact that my mask only makes a distinction with the key’s ‘starting’ pitch, the pitches within the octatonic scale and the rest of the pitches. The masks is very simple and likely does not properly convey to which degree each pitch supports an octatonic key.

From the 5th sonata, each sonata’s key starts getting more ambiguous

Throughout Scriabin’s career as a composer, you can quite clearly hear how his music starts getting more and more tonally ambiguous. Even to the point where it might not make much sense to discuss it in terms of tonality. If we attempt to create chromagrams of his later sonatas, we can see how this is indeed the case. The chromagrams of his 5th and 8th sonatas are quite the mess, notably when comparing them to the chromagrams of his 3rd and 4th.

Spotify is not very certain about the late sonata’s keys

Taking a look at the later half of Scriabin’s sonatas, it quickly becomes clear that Spotify has a harder time finding a key to stick with. Perhaps not unexpectedly so, since these pieces do not adhere to traditional tonality, as mentioned. While there might not be much of a point to trying to identify a key in these pieces, not to mention how Spotify’s analysis is probably not the best fit for unconventional music in general, it’s still interesting to see how it even fails to agree on a pitch across different recordings.

Perhaps different pianists might naturally bring forward different pitches throughout these kinds of pieces, consciously or not. Or perhaps this is a result of the recording process, where different equipment, studios and ‘finishing touches’ might alter the pitch content of a recording.

On top of that, the key confidence feature also shows that Spotify itself is aware of its uncertainty to some degree. I’ve mapped Spotify’s key_confidence feature to the opacity value of each recording. Some of the recordings are noticeably more opaque than the rest, but it’s difficult to tell what might have caused that other than the fact that these pieces are simply tonally ambiguous.

Aligning different performances

We previously saw how the 7th sonata seemed to be played quite differently when it came to tempo. We can look into this a little further using the recordings´ chromagrams. Besides showing us the pitch content of a recording, they can also be used to align different performances of the same music. To put it shortly, this is done by matching similar segments of the chromagrams.

The alignment itself can be seen in the dark blue diagonal line in this plot. We can for instance see how Letberg and Ashkenazy were playing the same musical fragment at around 300 and 270 respectively. As expected, the performers don’t quite seem to be in sync.

It is difficult to gather much information out of timbre

Similarly to the chromagrams, spotify also allows us to analyse the timbre content of the pieces, referred to as Cepstrograms. However, solo piano works don’t always lend themselves to this type of analysis, since you are generally limited in the kinds of timbres you can create with just a piano, unless you start employing extended techniques like in George Crumb’s case for instance. Regardless, there are instances where a pieces’s eccentricities are reflected in its cepstrogram.

Consider a sonata’s structure for example. One defining aspect of the sonata is how its first movement is composed in sonata form. Although like most musical conventions, composers have been straying from this practice over time, including Scriabin. Despite that, his first sonata does display a strong semblance of sonata form. When looking at the cepstrogram, you can make out how the exposition section is repeated once in the sonata’s beginning.

Structural details like this are hard to find in his other sonatas, especially when only looking at pitch and timbre. It is quite tough to define an overarching structure in his 8th sonata for instance. Nevertheless, you can clearly see the first 2 minutes being distinct from the rest of the piece. Upon giving this sonata a listen, you’ll notice that this section is noticeably unnerving yet tranquil.

The structure of Scriabin’s sonatas is not always easy to define

These plots display the pitch and timbre-based self-similarity within Scriabin’s first sonata. While the Timbre matrix does not seem to divulge much, since the overall timbre of the piece does not vary much, you can still make out how there’s one bar after around 150 seconds where there’s a large pause. Other than that, the plot does not tell us much.

The pitch-based self-similarity matrix on the other hand does clearly display the repetition of the sonata’s exposition. As seen in how there’s a dark blue line parallel to the diagonal of the matrix, indicating that two different sections of the piece overlap. The development and recapitulation are a bit harder to make out. However if you look closely, you can just barely see a very short dark blue diagonal line starting at 420 seconds on the x axis. This line is very short since the recapitulation almost immediately modulates to a different key.

Self-similarity matrices help very little with the later sonatas

As Scriabin started breaking away from conventional late-romanticism, he also began applying structure to his pieces in a different way. Rather than repetition, Scriabin utilizes many different motifs that he transforms, combines, transposes and puts into different contexts. What then defines the different sections of these pieces is how these motifs are applied. Two sections might be related through a motif appearing in a similar context, despite being modulated, in a different register or altered. This makes it very difficult to find any meaning these self-similarity matrices, as the sonatas’ structures are not defined by either pitch nor timbre. Not to mention how there’s is little to go off of on timbre. Let alone how these pieces’ actual structures are up to debate.

When specifically looking at the matrices of the eight sonata, we can once more see instances of prolonged silence in the timbre matrix. We can also once again clearly see the distinction of the introduction compared to the rest of the piece. Truthfully, once you start to thoroughly analyse this piece, you’ll find that it does follow some semblance of sonata form as seen here.

However it does not really show in either matrix, as expected. You can vaguely make out how the piece cycles through different harmonic material in the pitch matrix, although this is by no means a strong indication of its structure. For instance, what one could consider the exposition and recapitulation of this sonata, begin at 150 seconds and 550 seconds respectively, but this is not reflected in either plot. While these sections may be musically similar, they are harmonically quite different and the timbre matrix is generally only helpful in identifying silent parts in the recording. One small moment of silence can actually be seen preceding the aforementioned recapitulation at 550 seconds.

His early sonata’s often have an unstable pulse

We can also attempt to analyse a recording’s tempo by finding its most consistent frequency between onsets. The Spotify API does a lot of the heavy lifting by providing us with a recording’s onsets. Those can be used to make a Fourier-based tempogram which shows us at what frequency the onsets appear throughout the recording.

Some music is very suitable for these kinds of analyses, other music is a little less suitable. Our corpus unfortunately falls under the latter. Scriabin’s music is generally very free in expression and tempo, which was not uncommon for a late-romantic, or early modernist composer. It is therefore also not unexpected that performers play his music with a lot of rubato. You can see how the tempogram of the second movement of his third sonata is quite literally all over the place, which is in my opinion one of the more ‘rhythmically stable’ parts of his sonatas.

This is another problem apart from the natural use of rubato. The ‘beat’ in Scriabin’s music, like that of some of his contemporaries, is often very obscure to begin with. His frequent use of different polyrhythm and triplets, quintuplets or even septuplets, all contribute to creating rhythmically complex passages, To great effect in especially his later sonatas.

His late sonata’s crank it up a notch

As mentioned, this rhythmic complexity certainly does not let up in his later works. These pieces may sometimes show a vague pulse that you can follow, but that often quickly dissipates into a fluid texture. The appropriate use of rubato also certainly plays a role. The resulting tempogram therefore also does not tell us much.

Aside from all the previously mentioned qualities that these pieces share, it might also be worth mentioning the length of these pieces. While the late sonatas certainly are not among the longest of pieces, being only around 10 minutes long, they nonetheless contain quite a bit of material. When considering all factors, these pieces might also lend themselves to a greater variance in interpretation compared to, say, a late Scriabin prelude of only around a minute long. There simply is a lot more material for an experienced pianist to play around with, to put it one way. Regardless, this is also in a way simply a reinforcement of the fitting use of rubato and presence of complex rhythms.